Relay for personal interpreter

ABSTRACT

A relay is described to facilitate communication through the telephone system between hearing users and users who need or desire assistance in understanding voice communications. To overcome the speed limitations inherent in typing, the call assistant at the relay does not type most words but, instead, re-voices the words spoken by the hearing user into a computer operating a voice recognition software package trained to the voice of that call assistant. The text stream created by the computer and the voice of the hearing user are both sent to the assisted user so that the assisted user can be supplied with a visual text stream to supplement the voice communications. A time delay in the transmission of the voice of the hearing user through the relay is of assistance to the assisted user in comprehending the communications session.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation-in-part of application Ser.No. 09/288,420, filed Apr. 4, 1999, pending, which is a continuation ofapplication Ser. No. 08/925,558 filed Sep.8, 1997, now U.S. Pat. No.5,909,482.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] Not applicable.

BACKGROUND OF THE INVENTION

[0003] The present invention relates to the general field of telephonecommunications. In more particular, the invention relates to systems toassist telephone communications by those persons who are deaf, hard ofhearing, or otherwise have impaired hearing capability.

[0004] Most modern human communications in both social and businessenvironments takes place through sound communications. Yet within modernsociety there are many persons who have attenuated hearing capability.To assist those persons in making use of our telephonic communicationsystem built for the hearing majority, there has been developed a systemof telephone communication which has been principally used by the deafcommunity. That system makes use of a category of device known variouslyas a telecommunication device for the deaf (TDD), text telephone (TT) orteletype (TTY). Current TDDs are electronic devices consisting of a keyboard and a display as well as a specific type of modem, to acousticallyor directly couple to the telephone line. Modem TDDs permit the user totype characters into their keyboard, with the character strings thenencoded and transmitted over the telephone line to be displayed on thedisplay of a communicating or remote TDD device.

[0005] Most TDD communication is conducted in an idiosyncratic codespecific to the community of TDD users. This code, known as Baudot,evolved historically at a time when many telecommunication devices forthe deaf were based on mechanical or electromechanical devices ratherthan the current technology based on digital electronic components.Accordingly, the Baudot protocol was constructed for a set ofconstraints which are no longer relevant to present date devices. Theoriginal Baudot protocol was a unidirectional or simplex system ofcommunication conducted at 45.5 Baud. The conventional Baudot characterset was a character set consisting of 5 bit characters and the systemencodes the bits of those characters in a two-tonal system based oncarrier tones of 1400 and 1800 Hertz.

[0006] The system of TDD communications is widely used and in fact hasbecome indispensable to the deaf community throughout the industrializedworld. Deaf persons extensively communicate with their neighbors andwith other deaf and hearing people remotely, using the TDD system. Inaddition, systems have been developed to facilitate the exchange ofcommunication between the deaf community and hearing users who do nothave access to or utilize a TDD device. In the United States, telephonecompanies have set up a service referred to as a “relay.” A relay, asthe term is used herein, refers to a system of voice to TDDcommunication in which an operator, referred to as a “call assistant,”serves as a human intermediary between a hearing user and a deaf person.Normally the call assistant wears a headset that communicates by voicewith the hearing user and also has access to a TDD device which cancommunicate to the deaf user using a TDD appropriate protocol. In normalrelay operations in the prior art, the call assistant types at a TDDkeyboard the words which are voiced to her by the hearing user and thenvoices to the hearing user the words that the call assistant sees uponthe display of his or her TDD. The call assistant serves, in essence, asan interpreting intermediary between the deaf person and the hearingperson to translate from voice to digital electronic forms ofcommunication.

[0007] A limitation in the effectiveness of current relay protocols isthe necessity for the call assistant simply to type what is said.Typical call assistants can usually type reasonably well, typically inthe range of 40 to 60 words per minute. While systems exist which permitthe digital encoding of verbal communications at a faster rate, such ascourt stenography used in the legal system, such systems requireextensive special training and are impractical for the numerous callassistants required by the relay systems in operation today. Thislimitation on the speed of conversion from speech to digitalcommunications hampers the effective flow of communication in a relaycall between the hearing person and a deaf person, since most hearingpeople speak at a rate which is higher than most call assistants cantype. In addition, since conventional Baudot communications isunidirectional, the flow of conversation in a relay assistedcommunication session can be somewhat awkward. For example, first thehearing person must voice a statement or question. Then the callingassistant must type that statement or question, which is thentransmitted at Baudot speeds, which are slower than normal human voicecommunication, to the deaf person. The deaf person waits until theentire statement or question is transmitted to him or her, after whichhe or she composes a response and types it in at his or her TDD. Thenthe communication flows backward to the call assistant who must voice tothe hearing person what the deaf person has typed at his or herterminal. This process enables a degree of two-way communication betweena deaf person and a hearing person, but the system tries the patience ofthe hearing person, since it is typically not conducted at a paceanywhere close to normal human communications.

BRIEF SUMMARY OF THE INVENTION

[0008] The present invention is summarized in that a relay system tofacilitate the translation of information and communication between deafand hearing persons includes a call assistant who re-voices the words ofthe hearing person which are spoken to the call assistant. The wordsspoken by the call assistant are recognized by a speech recognitioncomputer program which has been trained to the voice pattern of the callassistant, such that the words are promptly translated into text andformatted into a high speed digital communication protocol. That highspeed digital communication message is then transmitted electronicallypromptly by telephone to a visual display accessible to the deaf person.

[0009] It is an advantage of the invention described herein that thecall assistant does not have to type most, if any, of the words spokenby the hearing person in the communication session so that the overallspeed of communications from the hearing person to the deaf person isdramatically increased.

[0010] It is an object of the present invention that the design andutilization of a relay operated in accordance with the protocolsdescribed herein permits the introduction of small hand-held personalinterpreter which will enable on the spot communications between deafpersons and hearing persons wherever the deaf persons might go.

[0011] Other objects, advantages and features of the present inventionwill become apparent from the following specification when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0012]FIG. 1 is a schematic block diagram of a re-voicing relay.

[0013]FIG. 2 is an exterior view of a personal interpreter enabled bythe relay of FIG. 1.

[0014]FIG. 3 is a schematic block diagram of the personal interpreter ofFIG. 2.

[0015]FIG. 4 illustrates the operation of a captioned telephone servicesupported by a relay.

DETAILED DESCRIPTION OF THE INVENTION

[0016] The present invention involves developments in the improvement ofcomputer assisted relay services. These developments are based uponusing voice recognition software, operated by a call assistant (a “CA”),to assist in the voice to text translation inherent in providing relayservices. The CA re-voices the words spoken by a hearing user, with thevoice recognition software translating the voice into text, so that nearto simultaneous voice and text of the voice can be delivered to anassisted user.

[0017] The re-voicing relay has its first implementation in providingvoice to text transcription services for the deaf. However, the relayvoice to text transcription service has use for users other than thosewho are deaf. It is envisioned that there are a number of hearing orpartially hearing users who would have reason to benefit from voice totext transcription services. For example, a device known as a captionedtelephone, described below, permits telephone users who have hearingdifficulty in use of the telephone by providing a near-simultaneous texttranscription of telephone conversations, the text being provided by arelay. Relay voice to text service might also be useful for anyapplication in which it is desired to supplement voice communications bya text transcription of the voice spoken on the telephone. Thus, whilethe operation of the relay will sometimes be described here by referringto an “assisted user,” who may be deaf or hard of hearing, but who alsomay be a normally hearing person who simply wants text assistance forsome reason. The user at the other end of the line will be referred tohere as the hearing user, simply for the purpose of having something tocall that person, even though both users may be hearing.

[0018] Introduction to Re-Voicing Relay.

[0019] The computer-assisted relay system is intended to provide moreconversation-like performance of voice to text interpreting fortranslating between an assisted user and a hearing user. The method ofoperating the relay described herein is applicable to the broad TDDcommunity, and also to all the applications in which a relay might benormally used. However, since the advantages of this system are mostclear in view of its usefulness in enabling the advent of the trulyportable personal interpreter for the deaf, a diversion to discuss whatthis device is and how the relay may enable its practical use isappropriate here.

[0020] Personal Interpreter.

[0021] Shown in FIG. 2 is an illustration of what a personal interpreter10 can look like. This would be a small hand held device typically thesize of a small hardbound book. It would have a keyboard of minimalsize, but useable by a deaf person who can type. It would have amulti-line display, but the display could be any size that convenientlyfits in the case of the device. The device would also have a key orswitch which would initiate its operation.

[0022] Shown in FIG. 3 is a schematic block diagram of the internalmechanics of the personal interpreter. The personal interpreter keyboardshown at 12 and its display as shown at 14. Inside the interpreteritself is a microprocessor shown at 16. Not shown, but included withinthe personal interpreter, would be the appropriate memory and interfacedevices so as to allow the microprocessor to be programmed and tooperate the personal interpreter and perform its functions, in a mannerwell known in the art. Also inside of the personal interpreter is amodem 18. The modem 18 is preferably a modem specifically designed forinterface with the deaf telecommunications system. Mosttelecommunications with the deaf community are conducted using a Baudottype code. It is preferred that the mode be designed to use the enhancedform of Baudot communication known as “Turbo Code” (Ultratec), which isgenerally described in US Pat. No. 5,432,837, U.S. Pat. Nos. 5,517,548,and 5,327,479, the disclosure of which is hereby incorporated byreference. It is even more preferred that the modem use a new variant ofTurbo Code, one which uses higher carrier frequencies (in the range of3000-3500 hertz) and a faster baud rate (over 100 baud). The output ofthe modem is preferably wired to a cellular telephone 20 included withinthe case of the personal interpreter 10. The cellular telephone 20 has asuitable antenna provided on it so that it may dial a cellular telephonenetwork by radio frequency communications of the type normally conductedby cellular telephones. The personal interpreter also includes jack 28to connect to a conventional wired or land-line telephone line as well.The personal interpreter also include a microphone 22 and a speaker 24.A filter 26 connects the speaker 24 and the microphone 22 to thetelephone 20.

[0023] A brief description of the operation and functionality of thepersonal interpreter reveals the dramatic improvement and convenienceand portability that this device gives to deaf people. A deaf user couldgo into an establishment, be it a government office or retail facility,in which there are only hearing persons. The deaf person would carrywith him or her the personal interpreter 10. The deaf person would thenplace the personal interpreter 10 upon a counter or other surface, openit up, and press the initiation key or start button. The microprocessor16 and modem 18 of the personal interpreter then power up and act inmany ways like a normal TDD device operating in telecommunicationstandard, such as Turbo code. However, there is one critical difference.The start or initiation key further causes the microprocessor 16 of thepersonal interpreter to dial a relay to set up a relay communicationsession and includes in its communication with the relay a message,using the enhanced command features available in advancedtelecommunication protocols, such as Turbo Code, to initiate a specialformat of relay call adapted for the personal interpreter. Other codeswhich permit command functions, such as ASCII or CC ITT, could also beused. The first operation is to activate the cellular telephone anddirect the cellular telephone to dial the number of a relay operating inaccordance with the method of the present invention. The cellulartelephone dials the relay. Obviously, no wired connection is required toallow the cellular telephone function to establish a telephoneconnection with the remote relay, but alternatively the jack 28 to aconventional telephone line could be used. In addition, when the relayanswers the telephone connection, the microprocessor 16 of the personalinterpreter 10 is instructed to provide command codes to the remoterelay. These command codes, a feature possible through the use of TurboCode, permits the personal interpreter to tell the relay that this is apersonal interpreter-type relay communication session. All of this canhappen in the time necessary to initiate the cellular call, perhaps twoto ten seconds.

[0024] Then, the deaf person can use the personal interpreter totranslate words spoken by hearing persons in the presence of thepersonal interpreter into visually readable text. This is done by thepersonal interpreter 10 through an unseen relay. Words spoken by thehearing persons in the presence of the personal interpreter 10 arepicked up by the microphone 22. Those words are then transmitted throughthe cellular telephone 20 to the remote relay. The relay, operating aswill be described below, then immediately transmits back, in enhancedTurbo Code, a digital communication stream translating the words thatwere just spoken. The words are received by the modem 18, and themicroprocessor 16 in the personal interpreter 10, and it is displayedpromptly upon the display screen 14. If the deaf person can speak, he orshe may then answer the hearing person with a spoken voice, or, the deafperson may alternatively type upon the keyboard 12. If the deaf usertypes on the keyboard 12, the personal interpreter transmits thecommunication by digital communication to the relay. The call assistantat the relay then reads and speaks the words typed by the deaf userwhich are transmitted to the speaker 24 contained in the personalinterpreter into a voice communication which can be understood by thehearing users. The filter 26 filters out the digital communicationfrequencies from the sound generated by the speaker 24. Thus, inessence, the deaf person has a personal interpreter available to him orher at all times of the day or night wherever the deaf person is withinthe range of the cellular telephone system. Also, because the relay ispreferably operating in accordance with the fast translation methodologydescribed below, a very conversation-like feel can occur in thecommunication session between the deaf user and the hearing persons inthe presence of the personal interpreter 10. In order for thiscommunication session to be satisfactory to the hearing users as well asthe deaf person, however, the relay must operate exceedingly rapidly. Itis, in part, to meet the need for the exceeding rapidity of thisconversational style of communication that the relay protocol of thepresent invention has been designed.

[0025] Re-Voicing Relay.

[0026] Shown in FIG. 1 is a relay intended to provide the voice to textcapability to support the personal interpreter, and which can alsosupport other voice to text services to provide services for assistedusers. FIG. 1 is intended to show, in schematic fashion, how such arelay system can be set up. Shown at 32 is a telephone of a hearingperson. Instead of a telephone of a hearing person, the input could alsobe the microphone of the personal interpreter 10 shown in FIGS. 2 and 3.The telephone of the hearing person 32 is connected through a telephoneline 34 to a voice input buffer 36 at the relay. The telephone line 34can be an actual physical land line, i.e. two pair between thetelephones, or can be a cellular or other over-the-air telephone linkageor can be an internet protocol digital connection. The voice inputbuffer 36 is a simple buffer to ensure temporary capture of the voice inthe event that the call assistant gets behind and needs to buffer ordelay the voice of the hearing person. In any event, the output of theinput voice buffer 36 is provided to a headset 40 where earphones 38produce the sound of the remote speaking person in the ear of the callassistant. The call assistant is wearing the headset 40 and sitting at acomputer 42 capable of communicating in an enhanced Baudotcommunication, such as Turbo Code or whatever other code protocol isbeing used. However, typically the call assistant does not type thewords which the call assistant hears in his or her earphone 38. Instead,the call assistant then speaks the words which he or she hears in theearphones 38 into a microphone 39 in the headset 40. The microphone 39on the headset 40 is connected to transmit the voice of the callassistant to the computer 42 at which the call assistant sits.

[0027] The computer 42 has been provided with a voice recognitionsoftware package which can recognize the spoken voice of the callassistant and immediately translate words spoken in that voice into adigital text communication stream. It is a limitation of currentlyavailable speech recognition software that the software must be trainedor adapted to a particular user, before it can accurately transcribewhat words the user speaks. Accordingly, it is envisioned here that thecall assistant operates at a computer terminal which contains a copy ofa voice recognition software package which is specifically trained tothe voice of that particular call assistant. It is also important thatthe voice recognition system be capable of transcribing the words of thevoice of the call assistant at the speed of a normal humancommunication. It has been found that a recently available commercialvoice recognition package from Dragon Systems, known as “NaturallySpeaking,” is a voice recognition software which will accomplish thisobjective and which will translate to digital text spoken words of auser at the normal speeds of human communication in conversation whenoperating on conventional modern personal computers. A voice recognitionsoftware system known as “Via Voice” from IBM provides similarfunctionality.

[0028] The computer terminal 42 of the call assistant then translatesthe text created by the voice recognition software to a modem 46 outthrough a telephone line 48 back to the display 50 located adjacent tothe assisted person. The display 50 can be a conventional TDD located atthe home of the remote assisted user, or can be the display 14 of thepersonal interpreter 10, or can be any other display or text capturedevice used by an assisted user.

[0029] For reasons that will become apparent, there is also a connectionfrom the microphone 39 of the headset 40 of the call assistant to theincoming telephone line 34 through a switch 52. The switch 52 canphysically be an electrical switch located between the microphone 39 andthe telephone lines 34 and the computer 42 or, as an alternative, it canbe a software switch operating in the computer 42 which passes the voiceof the hearing user through to the telephone lines as voice, or not,under conditions which are selected by the call assistant, by choices heor she makes at the keyboard 44 of the computer 42. The switch 52 isfunctionally a single pole double throw switch although, of course, ifthis function is performed by the computer it will be a logical not aphysical switch. In the simplest embodiment, the switch 52 is a simplesingle pole dual throw switch readily accessible to the call assistantwhich passes the voice of the call assistant from the microphone eitherout onto the telephone line 34 or to the computer 42.

[0030] It is a further enhancement to the operation of the relayconstructed in accordance with the present invention that the earphones38 have noise attenuating capability. Noise canceling earphones arecommercially available today or, for this purpose, the computer 42 canbe provided with noise canceling sound generation software which wouldcreate sound transmitted to the earphone 38 so as to cancel the soundsof the call assistant's own voice. The noise attenuation or cancellationavoids distracting the call assistant, since he or she would then beless distracted by the words that he or she has spoken, and thus wouldbe less likely to be distracted from the concentration of the task ofre-voicing the sounds of the voice heard in the call assistant's ear.

[0031] Similarly, another option which would be advantageous is that thesoftware providing for the creation of the digital text string by voicerecognition be buffered in its output flow to the modem 46. Before thecomputer 42 would pass the data on to the modem 46, the data would firstbe displayed on the computer screen of the computer 42 for review by thecall assistant. The purpose of this option would be to permit the callassistant to use the keyboard to spell or correct hard-to-spell words,or to create corrections of any misinterpretations created by the voicerecognition software, from the words spoken by the call assistant. It isanticipated that if such an option is utilized, it would require fairlyinfrequent use of the keyboard by the call assistant, since frequent usewould clearly slow down the through-put of the communications.

[0032] The relay of FIG. 1 can operate with normal TDDs or with apersonal interpreter as shown in FIGS. 2 and 3 and can also be used tosupport the operation of the captioned telephone described below. Ineither event, the hearing person speaks in the telephone 32 and thewords are transmitted through the telephone line 34 to the voice buffer36. The voice buffer 36, again operating under the control of the callassistant, would buffer the voice signals from the hearing user asneeded for the call assistant to keep up. The call assistant would hearthe voice of the hearing user through the ear piece 38 and then wouldre-voice those same words into the microphone 39. The words that thecall assistant speaks into the microphone 39 would be fed to thecomputer 42 where the voice recognition software, trained to the voiceof the call assistant, would translate those words into a digital textstream. The digital text stream would be turned into a digitalcommunication stream by the modem 46 and passed on the telephone line 48to a display 50 which can be observed by the assisted user. Experiencehas shown that using currently available technology the delay betweenthe time the hearing user speaks into the telephone 32 and the time thewords appear on the display 50 of the assisted user is a modest numberof seconds.

[0033] In the reverse, when a non-speaking assisted user types onto hisor her telecommunication device, the digital signals are transmitted tothe computer 42 which displays them for the call assistant who thenvoices those words into the microphone 39 which words are thentransmitted onto the telephone line 34. Note that the presence of theswitch 52 is therefore important in this mode. Since the voice of thecall assistant serves two different functions in the operation of thissystem, the signal on the call assistant's voice must be switched sothat the hearing user 32 only hears the voice for the communicationswhich are intended to be directed to that person. The switch 52 allowsfor the voice of the call assistant only to be directed to the hearingperson at the appropriate times.

[0034] Note that if the relay of FIG. 1 is used to facilitate atranslation based on a personal interpreter such as that shown in FIGS.2 and 3, there will be only one telephone line between the personalinterpreter and the call assistant. In essence, in a modification ofFIG. 1, the telephone 32 and the display 50 would both be within thepersonal interpreter 10. There would be only one telephone line, acellular link, between the personal interpreter 10 and the callassistant. This telephone line may be a virtual, as opposed to physicallinkage between the parties, as for example a linkage using internetprotocol over the internet. Note therefore that the voice of the callassistant and the digital communications created by the computer 42would then travel on that same telephone linkage to and from thepersonal interpreter 10. It is therefore important for this embodimentthat the personal interpreter 10 have appropriate filtering (i.e. thefilter 26) to filter out the digital communication carrier frequenciesof the digital communications protocol, so that they are not heard byhearing listeners in the presence of the personal interpreter 10. Thetelephone line must still carry voice signals, however, so that thespoken words articulated by the call assistant in response to digitalinstructions from the deaf user can be properly broadcast by the speakercontained within the personal interpreter.

[0035] The provision for filtering of the digital frequencies can bedone in any number of ways with two being the principal preferredmethodologies. If Turbo Code communications are conducted at theconventional Baudot frequencies of 1400 and 1800 Hertz, the personalinterpreter 10 could be provided with notch filters 26 to filter outsignals at those particular frequencies. It has been found that suchnotch filters still permit the transmission of audible andunderstandable human speech, even if they filter at those particularfrequencies. As an alternative, it is possible to change the Baudotfrequencies to those which are much higher, such as frequencies of 3000to 3500 Hertz. If this alternative is selected, the personal interpreter10 is then provided with a low pass filter which permits low frequencysounds to go to the speaker to be broadcast into the environment of thepersonal interpreter, while high frequencies are excluded.

[0036] It is also specifically envisioned that the filter of textcommunications signals from the voice signal can be done digitally orlogically rather than by analog filter. For example, it is possible toimplement the modem at site of the assisted user using a digital signalprocessing, or DSP, integrated circuit. Such a DSP chip can beprogrammed to recognize to separate voice signals from textcommunications signals and to pass along only the voice signals to thespeaker at the location of the assisted user. It is also contemplatedthat the telephone linkage between the relay and the assisted user couldbe a digital connection rather than an analog telephone line. Forexample, the linkage between the relay and the assisted user could be adigital wireless linkage or an internet protocol linkage, wired orwireless. In either of these instances, the appropriate protocol callsfor the transmission of packets of digital data, the packets beingmarked to indicate the type of information carried in the packets. Thepackets might contain, for example, voice signals which have beendigitized or might contain simple digital information representing thetext string of a conversation. In the instance of such a digital packettype communication protocol, the filtering out of the digital textinformation could be implemented simply by only converting the packetsmarked as voice back into sound. Packets marked as containing digitalinformation would be used to recover the text information for display tothe assisted user.

[0037] Captioned Telephone.

[0038] Shown in FIG. 4 is an illustration of how a typical telephonecall involving a captioned telephone would be set up. The hearing userat telephone 62 communicates through a telephone line 64 with the relay,indicated at 66. The relay, a re-voicing relay, communicates through atelephone line 68 with the assisted user. At the site of the assisteduser is a telephone 70 used by the assisted user and also a captionedtelephone device 72. The telephone 70 is conventional. The captionedtelephone device 72 is constructed to accomplish two objectives. Oneobjective is to filter, or separate, the digital signals carrying thetext information from the voice signal. The other objective is to takethe digital signals and create a visual display of the text informationfor the assisted user. This device is thus intended to assist the userto understand a greater portion of the conversation by providing avisually readable transcription of the text of the telephoneconversation so that the assisted user can read any words that he or shecannot hear properly.

[0039] While the utilization of the re-voicing relay is particularlyintended to be helpful for the personal interpreter and captionedtelephone applications, it is not intended to be limited to thoseparticular applications. For example, the voice to text capability ofthe re-voicing relay makes the use of such a relay attractive for somebusiness purposes, such as creating a text record of a businessnegotiation or interview session, conducted over or merely in thepresence of a telephone. If the call assistant is a simultaneoustranslator from one spoken language to another, the relay can be used toconduct language translations assisted by text transcriptions. The voicedelay relay can be used to present text nearly simultaneously to voicefor such applications.

[0040] In the implementation of a relay providing captioned telephoneservice, the relay transmits both a digital text message stream and thevoice of the hearing user over a telephone connection to the station ofthe assisted user. It is to be understood, however, that a conventionaltelephone single line connection is only one example of a telephonicconnection that can be used in this arrangement. Digital wirelessconnection, or PCS connection, or even internet protocol wired orwireless connection can be used to connect the relay to the assisteduser. The digital or analog nature of the telephonic connection is notcritical, the only criticality being that the connection is capable oftransmitting voice and text simultaneously from the call assistant tothat user.

[0041] It is to be understood that the present invention is not limitedto the particular illustrations and embodiments disclosed above, butembraces all such modified forms thereof as come within the scope of thefollowing claims.

I/we claim:
 1. A method of operating a relay system using a callassistant to facilitate communication between a hearing user and anassisted user by telephone comprising the steps of transmitting thevoice of the hearing user when speaking to the ear of the callassistant; the call assistant speaking in voice the same words that thecall assistant hears spoken by the hearing user into a microphoneconnected to a digital computer; the digital computer using voicerecognition computer software trained to the voice of the call assistantto translate the words of the voice spoken by the call assistant into adigital text message stream containing the words spoken by the callassistant; transmitting both the digital text message stream and thevoice of the hearing user by telephone connection to the assisted user;displaying the digital text message stream to a captioned telephonedisplay device within sight of the assisted person; and transmitting thevoice of the hearing user to the assisted user.
 2. A relay to facilitatecommunications between an hearing user and an assisted user, the relayoperated by a call assistant, the relay comprising a speaker connectedto receive voice communications from the hearing user and to transmitthat voice to the ear of the call assistant; a microphone connected topick up the voice of the call assistant; a digital computer connected tothe microphone, the computer programmed to use a voice recognitioncomputer software package to translate the words spoken by the callassistant into a digital text stream; and a telephonic connection totransmit both the digital text stream and the voice of the hearing userover a telephonic connection to the location of the assisted user.
 3. Amethod of operating a relay system using a call assistant to facilitatecommunication between a hearing user and an assisted user using a singletelephone connection connecting to a personal interpreter located in thepresence of both the hearing and assisted users, the hearing userspeaking words in voice, the method comprising the steps of transmittingthe voice of the hearing user when speaking over the single telephoneconnection to the ear of the call assistant; the call assistant speakingin voice the same words that the call assistant hears spoken by thehearing user into a microphone connected to a digital computer; thedigital computer using voice recognition computer software trained tothe voice of the call assistant to translate the words spoken by thecall assistant into a digital text message stream containing the wordsspoken by the call assistant; transmitting both the digital text messagestream and the voice of the hearing user over the single telephoneconnection back to the personal interpreter of the assisted user;displaying the digital text message stream by the personal interpreterwithin sight of the assisted person; and transmitting the voice of thehearing user to the personal interpreter which reproduces that voice forthe assisted user.